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ABSTRACT 


A  method  is  presented  for  reconstructing  the  geometry  of  a  road  from  a  sin¬ 
gle  image  of  the  road.  This  problem  has  an  infinity  of  solutions  unless  restrictive 
hypotheses  about  geometric  characteristics  of  this  road  are  assumed.  The  road  is 
modeled  as  a  space  ribbon  defined  by  a  spine  (centerline)  and  generators  which 
are  horizontal  line  segments  cutting  the  spine  at  their  midpoint  at  a  normal  an¬ 
gle.  Properties  of  two  neighboring  generators  of  such  a  ribbon  are  examined,  and 
it  is  found  that  if  a  generator  is  known,  a  neighbor  is  completely  defined  if  one  of 
its  ends  is  known.  The  proposed  method  uses  this  property  to  reconstruct  the 
visible  part  of  the  world  road,  by  iteratively  finding  a  series  of  generators.  For 
validation  of  this  method,  a  road  making  an  S  on  a  hill  or  in  a  valley  is  defined 
analytically  and  graphically,  and  its  perspective  image  is  obtained;  from  this  im¬ 
age,  algorithms  to  be  tested  reconstruct  a  ribbon  which  is  compared  to  the  origi¬ 
nal  world  model  by  superposition,  in  top  view  and  side  view.  The  proposed 
method  is  tested  against  a  simple  method  which  assumes  that  the  ground  is  flat 
(“flat  earth  assumption”),  and  against  another  method  which  uses  vanishing 
points. 


Th«  support  of  the  Defense  Advanced  Research  Projects  Agency  and  the  U.S.  Army  Engineering  Topographic  Labora¬ 
tories  under  Contract  DACA7S-8d-C-00(M  (DARPA  Order  5006)  is  greatfully  acknowledged. 


1.  Introduction 

In  a  vehicle  able  to  follow  a  road  without  human  intervention,  one  of  the  tasks  to  perform  is  the 
construction  of  a  world  model  of  the  road  from  the  images  given  by  the  vehicle's  sensors  [1,  2]. 
Sensors  being  currently  tested  for  this  task  are: 

-  A  single  camera  capturing  images  of  the  scene. 

-  A  pair  of  cameras  for  stereo  triangulation  between  two  images  of  the  scene. 

-  A  laser  ranger  giving  range  data  by  measuring  the  phase  lag  between  transmitted  and  reflected 
pulses  of  a  laser  beam  scanning  the  scene. 

We  will  not  discuss  the  relative  strengths  and  weaknesses  of  these  techniques  here,  but  will 
simply  remark  that  a  camera  will  most  probably  be  present  in  the  final  designs  of  autonomous 
navigation  systems,  in  association  with  another  camera  and/or  a  laser  ranger.  An  autonomous 
system  will  apply  a  multiple-expert  strategy,  obtaining  its  clues  from  several  groups  of 
algorithms,  each  group  processing  specifically  filtered  pieces  of  the  total  information  received  by 
the  different  sensors.  One  of  these  experts  could  focus  on  the  information  obtained  by  a  single 
camera.  In  this  paper  we  present  a  method  which  such  an  expert  system  could  invoke  to  build 
geometric  models  of  the  world,  in  a  limited  world  of  roads  and  highways. 

Papers  such  as  [1],  [3],  [4]  also  consider  the  problem  of  road  inverse  perspective,  and  the 
methods  proposed  there  are  based  on  finding  pairs  of  line  segments  which  are  images  of  parallel 
segments  in  the  scene.  For  such  pairs,  the  direction  from  the  focal  point  of  the  camera  to  the 
vanishing  point  of  the  pair  gives  the  direction  of  the  parallel  segments  in  space  (Figure  1). 


Furthermore,  the  intersections  of  these  parallel  segments  with  the  segments  processed  at  the 
previous  iteration  can  be  found  as  the  projections  of  the  corresponding  image  intersections  onto  the 
plane  of  the  previous  segments.  These  parallel  segments  are  then  fully  defined  in  space. 

This  type  of  procedure  relies  on  the  assumption  that  points  which  are  intersections  of  the  edges 
with  a  plane  normal  to  the  centerline  ("opposite  points")  are  points  at  which  the  tangents  to  the 
edges  are  parallel.  It  also  assumes  that  the  line  segments  found  by  image  processing  around  the 
images  of  these  points  are  images  of  parallel  line  segments  ("opposite  segments").  One  problem 
with  this  method  is  that  the  test  for  opposite  points  and  opposite  segments  can  be  applied  only  in 
3-D.  Thus,  checking  whether  two  segments  in  the  image  are  images  of  opposite  segments  in  space 
would  require  knowing  the  3-D  structure  of  the  world  road  that  we  are  trying  to  calculate. 

A  second  problem,  which  occurs  in  images  of  road  turns,  is  that  the  set  of  line  segments  found 
by  image  processing  to  approximate  the  images  of  the  edges  of  the  road  is  a  discontinuous 
representation  of  continuous  curves,  and  it  is  probable  that  none  of  the  possible  pairs  formed  by 
picking  a  segment  on  each  edge  is  the  image  of  a  pair  of  parallel  segments.  If  Figure  2  shows  the 
correct  inverse  perspective  world  segments  of  the  segments  in  an  image  (no  pair  of  parallel 
segments  can  be  found),  then  a  method  assuming  that  parallel  pairs  of  world  segments  can  be 
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found  will  give  the  wrong  inverse  perspective  segments  and  an  erroneous  world  road. 


In  this  case  a  pair  of  parallel  world  segments  built  from  any  pair  of  image  segments  will  diverge 
from  the  real  road. 

A  third  problem  is  that  a  road  reconstructed  using  parallel  pairs  of  edge  segments  is  developable 
because  it  is  a  chain  of  planar  patches.  Such  a  model  is  acceptable  for  flat  roads  or  straight  roads 
on  a  changing  slope,  but  not  when  a  turn  occurs  on  a  slope.  In  such  combinations  of  turn  and 
slope,  the  premise  that,  for  opposite  points,  tangents  to  the  road  edges  are  parallel  •  the  very  basis 
for  the  vanishing  point  methods  -  is  wrong.  In  such  cases,  road  patches  made  from  opposite 
segments  are  warped,  and  trying  to  model  them  with  planar  patches  gives  a  reconstructed  road 
which  does  not  match  the  real-world  road. 
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Figure  3 

This  last  point  is  illustrated  in  Figure  3.  A  road  climbs  a  constant  slope,  and  takes  a  turn  while 
its  centerline  stays  in  the  plane  of  the  slope.  For  a  real  world  road,  a  section  of  the  hill  would  look 
like  the  top  right  drawing. 

The  only  possible  developable  road  taking  a  turn  on  a  constant  slope  is  a  planar  road  which  stays 
on  the  surface  of  the  hill,  and  takes  an  unrealistic  bank  equal  to  the  slope  of  the  hill  (bottom 
drawing).  The  image  of  such  a  road  would  be  much  wider  after  the  turn  than  the  image  of  a 
non-developable  road  (top);  therefore  an  inverse  perspective  program  using  the  top  perspective 
view  and  reconstructing  the  road  with  planar  patches  will  not  find  a  road  on  the  slope  of  the  hill. 
From  the  narrowing  of  the  road  after  the  turn  in  the  top  perspective  view  of  the  figure,  such  a 
program  will  assume  that  the  road  after  the  turn  is  much  further  away  than  it  actually  is. 

The  previous  comments  have  shown  the  importance  of  a  geometric  model  of  road  ribbons 
flexible  enough  to  represent  the  situations  encountered  in  the  real  world.  The  following  section 
summarizes  the  proposed  modeling  scheme. 
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2.  Summary  of  the  world  road  modeling  scheme 

In  the  inverse  perspective  algorithm  presented  in  this  paper,  the  world  road  ribbon  is  defined  by 
a  spine  and  a  sweeping  line  segment  which  slides  along  the  spine  at  its  centerpoint,  while  remaining 
normal  to  this  spine.  Furthermore,  the  generating  segment  is  assumed  to  remain  horizontal.  This 
road  model  was  originally  suggested  by  Ozawa  and  Rozenfeld  in  [5]. 

This  defiivition  of  the  road  ribbon  is  continuous.  In  practice,  however,  only  a  discrete  number  of 
neighboring  cross-segments  are  computed,  and  we  discuss  in  Sections  3  and  4  the  geometric 
properties  of  two  successive  cross-segments  in  a  space  road  ribbon.  In  the  proposed  algorithm,  the 
calculation  of  a  new  cross-segment  is  iterative,  requiring  knowledge  of  the  previous  cross-segment. 
The  geometric  relations  between  two  successive  cross-segments  are  used  to  deduce  the  new 
cross-segment.  These  are  summarized  in  the  four  following  characteristics  describing  a  new 
cross-segment: 

1.  The  new  cross-segment  is  normal  to  the  road  centerline.  In  a  discrete  representation  of  nearby 
cross-segments,  this  can  be  expressed  by  saying  that  the  average  direction  between  the  previous 
cross-segment  direction  and  the  new  cross-segment  direction  is  normal  to  the  vector  joining  the 
midpoints  of  these  two  segments. 

2.  The  new  cross-segment  has  the  same  length  as  the  previous  one. 

3.  The  new  cross-segment  is  horizontal. 

4.  The  images  of  the  ends  of  the  new  cross-segment  are  on  the  images  of  the  road  edges. 

These  four  constraints  usually  yield  more  than  one  possible  new  cross-segment.  Among  these, 
the  cross-segment  which  gives  the  minimum  change  of  road  slope  is  chosen. 

Sections  3  and  4  show  that  the  first  two  constraints  constrain  a  pair  of  neighboring 
cross-segments  to  form  a  warped  isosceles  trapezoid.  In  Section  5,  details  of  the  calculation  are 
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given  for  combining  this  warped  isosceles  trapezoid  approximation  with  constraints  3  and  4  and 
finding  a  new  horizontal  cross-segment  from  the  knowledge  of  the  road  image  and  a  previous 
cross-segment.  Section  6  completes  the  description  of  this  road  reconstruction  algorithm  by 
presenting  a  method  to  obtain  an  initial  cross-segment.  Section  7  applies  this  algorithm  to  the 
perspective  image  of  a  road  defined  analytically  which  makes  an  S  on  a  hill  or  in  a  valley.  The 
results  are  compared  to  the  reconstructions  obtained  by  a  method  that  assumes  that  the  ground  is 
flat,  and  by  another  method  that  tries  to  find  images  of  parallel  segments  in  the  perspective  image. 

3.  Geometry  of  a  road  of  constant  width 

3.1.Planar  road 

The  only  reason  for  first  considering  a  planar  road  is  to  introduce  the  concepts  which  will  be 
applied  to  a  non-planar  road.  The  properties  of  a  space  ribbon  will  of  course  remain  appropriate  for 
modeling  a  planar  road. 

Consider  a  planar  road  of  constant  width  (Figure  4).  It  has  a  ribbon  shape  which  tan  be 
generated  in  the  foUowi’- way: 

a.  A  planar  spine  defines  the  trajectory  of  the  road  in  the  plane. 

b.  A  line  segment  of  constant  length  has  its  midpoint  on  the  spine,  and  is  required  to  stay  normal  to 
the  spine  as  it  moves  along  the  spine.  The  ribbon  generated  by  this  process  is  a  Brooks  ribbon  of 
constant  width  [6]. 

Such  a  ribbon  makes  a  reasonable  model  of  a  planar  road  of  constant  width: 

1.  The  sides  of  the  road  are  the  loci  of  the  end  points  of  the  generating  segment. 

2.  The  spine  is  the  centerline  of  the  road. 

3.  The  constant  length  of  the  generators  (cross  segments)  is  the  road  width.  The  end  points  of  a 
cross-segment  are  opposite  points  of  the  road. 


7 


spine 


Figure  4 


The  coordinates  of  a  point  L  moving  on  the  spine  can  be  expressed  as  functions  of  the  curvilinear 
abscissa  s.  The  spine  is  then  defined  by  the  coordinates  xL(s)  and  yL(s)  of  the  vector  function 
OL(s),  or,  omitting  the  origin  O,  L(s).  (We  use  bold  face  for  points,  vectors  and  vector  functions, 

and  plain  face  for  scalars).  The  unit  vector  tangent  to  the  spine  at  the  abscissa  sj^  is  the  derivative 

dL/  ds  for  the  value  s^.  If  AB  is  the  cross-segment  at  this  point,  we  can  express  the  fact  that  AB  is 
normal  to  the  spine  by  the  equation 

AB‘dL/ds  =  0  for  s  =  si  (1) 

Instead  of  dealing  with  an  infinity  of  contiguous  cross-segments,  we  can  capture  the  general 
form  of  the  road  ribbon  from  a  discrete  set  of  closely  spaced  cross-segments,  in  the  same  way  as  a 
railroad  can  be  defined  by  its  ties. 

This  discrete  approximation  is  more  useful  for  iterative  computation  purposes  than  a  continuous 
representation;  therefore  we  will  examine  some  of  its  properties. 

Consider  two  successive  cross-segments  AB  and  A'B',  and  their  midpoints  L  and  L'  (Figures  4, 
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5).  We  are  looking  for  a  discrete  equivalent  of  the  property  that  cross-segments  are  normal  to  the 
centerline.  An  appropriate  approximation  is 

1/2  (AB  +  A’B')  •  LL’  =  0  (2) 

Indeed,  if  we  knew  the  curvilinear  Sj  and  s’j  for  L  and  L',  we  would  write  the  previous  equation 
as 

1/2  (AB  +  A'B')  •  (L'-  L)  /  (s'l  -  s^)  =  0  (3) 

and  this  equation  tends  to  equation  (1)  when  the  point  L'  tends  toward  the  point  L,  which  supports 
our  choice  of  equation  (2)  in  the  discrete  approximation  of  the  road  (Other  choices  which  do  not 
give  symmetrical  roles  to  AB  and  A'B',  or  use  more  than  two  cross-segments,  are  possible).  This 
equation  simply  states  that  the  average  direction  of  the  road,  LL',  is  normal  to  the  average 
direction  (AB  +  A'B')  /  2  of  the  two  cross-segments  AB  and  A'B'  located  at  L  and  L'. 


Figure  5 
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Consider  the  line  segment  IJ  joining  the  midpoints  I  and  J  of  the  line  segments  A  A'  and  BB'. 
The  vector  U  is  equal  to  (AB  +  A'B')/2: 

(AB  +  A'B')/2  =  1/2(AI  +  U  +  JB  +  A'l  +  U  +  JB')  =  U  (4) 

Thus  equation  (2)  also  states  that  U  is  normal  to  LL’. 

Consider  the  line  segment  joining  A  and  A',  and  the  angles  between  AB  and  AA'  and  between 
A'B'  and  AA'.  The  cosines  of  these  angles  are  obtained  from  the  dot  products  AB»AA'  and 
A'B'«A'A: 


AB  •  AA*  -  A'B'  •  A'A  =  AA'  •  (  AB  +  A'B') 

=  (  AL  +  LL'  +  L'A')  •  (  AB  +  A'B') 

Since  LL'  •  (  AB  +  A'B')  =  0  and  AL  =  AB/2  and  L'A'  =  -  AB/2,  this  simplifies  to 

AB  •  AA'  -  A'B'  •  A'A  =  1/2(AB  -  A'B')  •  (  AB  +  A'B') 

AB  •  AA' .  A'B'  •  A'A  =  1/2  (AB^  .  A'B'^) 

AB  •  AA'  -  A'B'  ‘A'A  =0  (5) 

Thus  the  angles  (AB,  AA')  and  (A'B',  A'A)  are  equal,  and  the  quadrilateral  ABB' A'  is  a  planar 
isosceles  trapezoid.  Therefore  the  line  segment  U  is  normal  to  AA'  and  BB'  (and  LL',  as  already 
found).  We  will  find  this  property  to  be  conserved  in  the  case  of  a  non-planar  road,  despite  the  fact 
that  in  that  case  AA',  LL',  and  BB'  are  not  generally  coplanar. 


3.2.  Non-planar  road 


In  the  analytic  representation  of  a  non-planar  road,  we  shall  retain  the  modeling  by  a  ribbon  such 
as  a  Brooks-type  ribbon;  a  spine  will  define  the  meandering  up  and  down  and  sideways  of  the 
road,  while  a  generating  line  segment  of  constant  length  will  give  some  flesh  to  that  spine  by 
sweeping  along  the  spine.  The  sliding  point  of  the  generating  segment  is  its  midpoint,  and  the 
segment  is  restricted  to  be  normal  to  the  spine  at  this  sliding  point.  For  a  spine  in  space,  however. 
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this  last  restriction  only  constrains  the  segment  to  be  in  a  plane  normal  to  the  spine  at  the  sliding 
point,  and  does  not  define  a  unique  ribbon:  the  generating  segments  could  take  any  angles  within 
these  normal  planes  (Figure  6).  The  intersection  of  the  normal  plane  with  a  horizontal  plane  can  be 
used  as  a  reference  for  measuring  these  angles.  This  reference  line  is  therefore  both  horizontal  and 
normal  to  the  spine.  The  angle  of  the  generating  segment  with  this  horizontal  line  in  the  normal 
plane  is  called  the  bank  or  superelevation  of  the  road.  Its  role  in  real-world  roads  is  to  eliminate 
rain  water  from  the  road,  and  to  give  a  favorable  horizontal  road  reaction  component  on  vehicles  in 
turns.  The  tangent  of  the  bank  is  ordinarily  3  %,  and  at  most  10  %  [5] . 


Let  us  assume  first  that  some  rule  is  used  to  define  a  unique  bank  for  each  generating  segment  as 
function  of  the  local  characteristics  of  the  spine,  and  that  a  unique  ribbon  is  therefore  generated  for 
any  given  spine,  in  which  the  generating  segments  are  normal  to  the  spine  at  their  midpoints.  In  a 
development  identical  to  that  followed  for  a  planar  road,  a  sliding  point  L  on  the  spine  can  be 
defined  as  a  vectorial  function  L(s)  of  a  curvilinear  abscissa  s.  The  cross-segment  AB,  whose 
midpoint  is  L,  is  normal  to  the  spine  in  L.  This  is  expressed  by  equation  (1): 

AB»dL/ds  =  0  (1) 
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If  we  now  want  to  define  the  space  road  ribbon  by  a  discrete  number  of  generating  segments, 
the  same  approximation  as  for  the  planar  road  applies  for  two  nearby  generating  segments  AB  and 
A'B'  (Figure  7): 

1/2  (AB  +  A'B’)  •  LL'  =  0  (2) 

We  showed  that  for  a  planar  road  this  relation  constrains  the  four  points  A,  B,  B',  A'  to  define  a 
trapezoid.  In  the  case  of  a  space  road,  the  segments  AB  and  A'B'  are  most  likely  to  be 
non-coplanar  due  to  combinations  of  slope  and  turns,  even  if  the  bank  is  assumed  to  be  constant  or 
null.  Consider  for  instance  the  ribbon  generated  by  the  edges  of  the  steps  in  a  spiral  stair  case.  The 
spine  is  a  cylindrical  helicoid.  All  the  generating  step  edges  are  horizontal  (zero  bank),  but  two 
consecutive  step  edges  AB  and  A'B'  are  non-coplanar,  because  the  line  segment  BB'  on  the  inside 
of  the  turn  is  much  more  vertical  than  the  segment  AA'  on  the  outside  of  the  turn;  the  quadrilaterals 
formed  by  two  consecutive  step  edges  are  warped.  A  similar  situation  occurs  when  a  change  of 
altitude  occurs  in  a  road  turn.  In  other  words,  such  a  ribbon  is  not  a  developable  surface,  and 
efforts  to  model  it  with  non-triangular  planar  patches  would  be  a  waste  of  time. 


Let  us  examine  some  geometrical  properties  of  a  warped  quadrilateral  ABB'A'  of  two  nearby 
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generating  segments  AB  and  A'B'. 

The  angles  (AB,AA')  and  (A'A,A'B')  are  found  to  be  equal,  using  the  previous  development  (5) 
to  show  that  the  quantity  AB»AA'  -  A'B'«A'A  is  null.  The  angles  (BA,BB')  and  (B'B,B'A') 
can  also  be  shown  to  be  equal  in  a  similar  way.  Thus  we  will  call  the  quadrilateral  ABB'A'  a 
warped  isosceles  trapezoid.  Note  that  such  a  figure  is  noi  obtained  by  folding  a  planar  isosceles 
trapezoid  along  one  of  its  diagonals,  since  this  procedure  would  decrease  the  values  of  the  two 
angles  at  the  diagonal,  while  leaving  the  other  two  angles  the  same,  and  the  angle  equalities  would 
be  lost.  Instead,  fold  a  parallelogram  ABA'B'  along  one  diagonal,  for  instance  AA'  (Figure  8). 

Warped  trapezoid  ABB'A': 

From  triangle  ABA', 
complete  parallelogram  ABA'B', 
then  rotate  triangle  AA'B' 
around  AA' 


B  A 

Figure  8 

The  maximum  folding  gives  a  planar  isosceles  trapezoid  ABB'A'  (the  order  of  the  points  A'  and  B' 
is  reversed  because  we  now  see  the  back  half  of  the  parallelogram).  For  all  incomplete  foldings, 
ABB'A'  is  a  warped  isosceles  trapezoid  (including  the  starting  parallelogram  itself). 

The  midpoints  of  AB  and  A'B'  are  L  and  L'  (Figure  7).  Calling  I  and  J  the  midpoints  of  AA' 
and  BB',  K  the  midpoint  of  U  and  K'  the  midpoint  of  LL’,  one  can  find  four  expressions  for  KK' 
and  add  them  to  show  that  K  and  K'  are  confounded.  Therefore,  in  a'warped  isosceles  trapezoid, 
I,  J,  L,  L',  and  K  are  coplanar  (we  will  not  use  this  property  in  this  paper). 

We  determine  that  U  is  normal  to  LL'  by  the  same  equalities  leading  to  equation  (4)  used  in  the 
planar  case. 

Next  we  want  to  show  that  U  is  also  normal  to  AA'  and  BB',  even  though  AA',  BB',  and  LL' 


are  not  parallel  in  the  general  case: 

AA'  •  IJ  =  (AL  +  LL'  +  L'A')  •  IJ 

AA'  *13  =  (AL  +  L'A')  •  IJ  since  U  is  normal  to  LL' 

AA'  •  U  =  (AL  -  A'L')  •  (AL  +  A'L')  from  eq.(4)  for  U. 

AA' •U  =  AL2-A'L'2  =  0  (6) 

4.  Warped  isosceles  trapezoids  in  zero-bank  space  ribbons 

In  the  following,  we  will  assume  the  bank  of  all  cross-segments  to  be  null  (Figure  9). 
Incorporating  a  non- zero  bank  in  the  geometric  modeling  of  the  road  would  be  possible,  but  it  does 
not  seem  that  rigid  rules  are  used  for  bank  in  road  construction,  in  part  because  an  optimal  bank  in  a 
turn  depends  not  only  on  turn  radius,  but  also  on  the  expected  speeds  of  the  vehicles. 
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With  the  hypothesis  of  zero  bank  for  a  ribbon  defined  by  a  continuum  of  generating  segments, 
for  any  given  point  L  of  the  road  spine,  the  generating  segment  is  at  the  intersection  of  the  plane 
normal  to  the  spine  at  this  point  L,  and  of  the  horizontal  plane  through  this  point.  An  equivalent 
statement,  using  vectors  only,  is  more  useful  for  computations.  We  define  a  vertical  vector  V.  The 
tangent  unit  vector  to  the  spine  at  L  is  dL/ds  (using  the  same  definitions  and  notations  as  for  the 
planar  horizontal  road).  The  direction  of  a  generating  segment  at  L  for  the  zero  bank  space  road  is 
then  given  by  the  cross  product  V  x  d  L/ds. 

In  the  case  of  a  definition  of  the  ribbon  from  discrete  cross-segments,  such  as  AB  and  A'B', 
equation  (4)  implies  that  if  AB  and  A'B'  are  horizontal,  then  IJ  is  horizontal  too: 

V  •  IJ  =  1/2  V  •  (AB  +  A'B')  =  0  (7) 

Therefore  U  is  normal  to  both  V  anJ  AA',  so  that  its  direction  is  given  by  the  cross  product 
VxAA'. 

Its  length  is  also  known:  since  IJ  is  normal  to  AA'  and  BB'  (equation  6),  I  and  J  are  the  projections 
of  A  and  B  on  the  line  in  the  direction  just  found  going  through  I;  thus  the  length  of  U  is  defined  by 
the  dot  product  of  AB  by  the  unit  vector  in  that  direction. 

This  unit  vector  is  (V  x  AA')/((  V  x  AA')^)*^.  Therefore  the  expression  for  IJ  is 

U  =  [(AB  •  (V  X  AA')/  (V  X  AA')  2]  (V  x  AA')  (8) 
where  the  part  in  brackets  is  a  scalar  resulting  from  the  calculation  of  the  length  of  IJ  and  the 
normalization  of  (V  x  AA');  the  last  parenthesis  is  the  vector  (V  x  AA')  which  gives  the  direction 
of  U. 

Now  that  U  is  known,  B'  is  uniquely  defined  from  AB,  A',  and  U,  by  equation  (4),  which 
gives 

B'  =  A' .  AB  +  2  [(AB  •  (V  x  AA')/  (V  x  AA')  2]  (V  x  AA’)  (9) 

This  relation  uniquely  defines  the  end  B'  of  a  cross-segment  A'B'  when  the  end  A'  is  given  and  a 
neighboring  cross-segment  AB  is  known.  It  will  allow  us  to  find  A'B'  from  AB  in  the  following 
section. 
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5.  Inverse  perspective  zero-bank  algorithm 

The  processing  of  the  persective  image  of  the  road  is  assumed  to  output  two  ordered  lists  of 
points,  one  for  each  side  of  the  road  image  (for  details  on  image  processing  procedures,  see  [7], 
[8]).  These  are  inputs  to  the  algorithm.  The  output  of  the  algorithm  is  an  ordered  list  of 
neighboring  cross-segments  of  the  world  road. 

Since  the  algorithm  reconstructs  a  world  road  cross-segment  from  a  previously  calculated 
cross-segment,  an  initial  cross-segment  is  needed  to  start  the  iteration.  Methods  for  obtaining  the 
initial  cross-segment  are  discussed  in  the  next  section.  We  assume  for  now  that  such  a 
cross-segment  has  been  found,  and  we  are  going  to  look  at  one  intermediate  step  of  the  iteration. 

The  two  edges  of  the  road  image  are  used  in  a  different  way  in  the  search  for  a  new 
cross-segment: 

-  From  one  edge  of  the  road  image,  we  pick  a  point ,  and  we  say  that  this  point  is  the  image  of  one 
end  of  our  yet  unknown  cross-segment;  this  end  is  somewhere  on  the  line  between  the  camera  focal 
point  and  the  chosen  image  point 

-  From  the  other  side  of  the  road  image  we  consider  the  segments  joining  consecutive  points  of  the 
road  side  image.  The  image  of  the  other  end  of  the  world  road  cross-segment  must  belong  to  one  of 
these  image  segments:  we  constrain  this  end  to  belong  to  one  of  the  planar  sectors  defined  by  the 
focal  point  and  the  image  segments. 
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Figure  10 

The  inverse  perspective  of  an  image  point  a’  of  one  road  image  edge  (Figure  10)  is  the  line  Oa' 
(O  is  the  focal  point).  The  inverse  perspective  of  a  segment  ef  of  the  other  road  image  edge  is  the 
planar  facet  between  the  lines  Oe  and  Of,  and  the  inverse  perspective  of  the  whole  edge  is  the 
polyhedral  cone  defined  from  the  facets  corresponding  to  each  segment.  One  end.  A',  of  the  road 
cross-segment  A'B'  belongs  to  the  line  Oa';  the  other  end,  B',  has  to  belong  to  the  polyhedral 
cone. 

If  A'  belongs  to  line  Oa',  vector  A’  can  be  expressed  as  a  function  of  a  single  parameter  m: 

A'  =  m  a'  (10) 

The  previous  section  showed  that  vector  B'  is  defined  as  a  function  of  A'  by 

B'  =  A'  -  AB  +  2  [(AB  •  (V  x  AA')/  (V  x  AA’)  2]  (V  x  AA')  (9) 

Here  A  and  B  are  the  end  points  of  the  previous  cross-segments  and  have  known  coordinates,  and 
V  is  the  unit  vertical  vector.  Thus  B'  is  a  function  of  a  single  parameter,  m,  and  the  locus  of  the 
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point  B'  is  a  space  curve.  Among  the  points  of  this  curve,  the  points  which  are  acceptable  as 
possible  ends  of  the  new  cross-segment  are  the  points  whose  images  are  on  the  road  edge  image. 
An  acceptable  point  B'  is  an  intersection  of  the  space  curve  locus  and  one  of  the  planar  facets 
defined  by  the  segments  of  the  edge  image  and  the  camera  focal  point.  We  do  not  need  to  calculate 
this  locus;  we  need  only  to  calculate  the  parameter  m  corresponding  to  these  intersections.  This  can 
be  done  as  follows: 

If  B'  belongs  to  a  planar  facet  such  as  defined  by  the  image  segment  ef,  we  can  write  that  the 
vector  B'  is  normal  to  the  normal  to  a  normal  n  to  this  plane: 

n  =  exf,  B'»n  =  0,  (11) 

We  must  specify  that  B'  is  not  only  in  the  plane,  but  on  the  limited  sector  between  the  lines  Oe 
and  Of,  and  in  front  of  the  camera.  These  two  conditions  are  expressed  by 

(B'xe)*(B'xf)<0.  (B'»e)>0  (12) 

When  both  sides  of  equation  (9)  are  multiplied  by  n  (dot  product),  a  scalar  equation  is  obtained, 
and  the  left  end  side  is  0,  since  B' .  n  =  0  (eq.  1 1). 

Replacing  A'  by  ma',  a  cubic  equation  in  the  parameter  m  is  obtained: 

a3m^+a2ni^  +  a2m  +  aQ  =  0  (13) 

and  the  four  coefficients  are  found  to  be 

a3  =  ( a'  •  n  )  ( V  X  a')  ^ 

a2  =  -  2  (a'  •  n)  ((V  x  a')  •  (  V  x  A))  -  (AB  •  n)  ( V  x  a')  ^ 

+  2(n»(Vx  a'))(  AB  •  (V  x  a')) 
ai  =  (a'  •  n)  (  V  X  A)  2  +  2  (AB  •  n)  ((V  x  a’)  •  ( V  x  A)) 

-  2  (n  •  (V  X  a'))  (AB  •  (V  x  A))  -  2  (n  •  (V  x  A))  ( AB  •  (V  x  a’)) 
ao  =  (AB  •  n)  ( V  X  A)  2  +  2  (n  •  (V  ? .  A))  (AB  •  (V  x  A))  ( 14 ) 
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The  real  roots  (one  or  three)  of  the  equation  are  easily  obtained  analytically.  These  values  of  m 
define  the  possible  positions  of  the  point  A'  on  line  Oa*  (from  condition  (10))  such  that  B'  satisfies 
condition  (9)  and  condition  (11).  Once  A'  is  found,  B'  is  uniquely  determined  by  (9).  For  the 
pair(s)  (A'B')  given  by  the  root(s)  m  of  (14),  only  the  pair(s)  for  which  B'  satisfies  (12)  is  (are) 
valid.  If  more  than  one  point  B'  satisfies  this  criterion  (12),  only  the  point  B'  such  that  the 
corresponding  road  direction  has  the  minimum  reasonable  slope  difference  with  respect  to  the 
previous  road  direction  is  kept. 

These  conditions  reduce  to  one  or  none  the  number  of  acceptable  solutions  for  A'B'  for  the 
image  segment  ef  being  considered.  If  no  acceptable  solution  is  found  with  the  image  segment  ef, 
the  next  image  segment,  fg,  is  used.  Only  a  small  number  of  such  image  segments  (2  to  4)  usually 
need  to  be  considered.  If,  after  a  limited  number  of  segments  are  examined,  no  solution  is  found, 
this  may  be  due  to  the  fact  that  the  inverse  perspective  of  a  road  image  with  polygonal  sides  cannot 
have  a  smooth  constant  width.  The  length  of  A'B'  is  increased  by  a  small  percentage,  and  the 
search  process  is  started  again  with  the  same  segments. 

6.  Acquisition  of  an  initial  cross-segment 

Since  the  process  described  above  is  iterative  and  uses  a  previously  calculated  cross-segment,  an 
initial  cross-segment  is  needed  to  start  the  algorithm  for  each  new  image  frame.  This  initial 
cross-segment  can  be  calculated  from  a  "flat  earth  assumption".  In  most  autonomous  systems,  the 
position  of  the  camera  is  known  with  respect  to  the  road  patch  above  which  it  operates;  this  initial 
road  patch  is  assumed  to  be  planar,  and  the  equation  of  its  plane  can  be  calculated  in  the  camera 
coordinate  system  as  a  function  of  the  camera  height  and  tilt  (it  is  invariant  to  camera  pan).  With  the 
proper  camera  tilt  and  camera  field  of  view,  the  line  segments  of  the  image  road  edges  at  the  bottom 
of  the  image  correspond  to  world  road  edges  which  are  close  to  the  camera,  and  belong  to  this 
plane.  These  world  segments  are  obtained  by  projecting  the  bottom  image  segments  of  the  left  and 
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right  road  edge  image  on  this  plane  (Figure  11). 


MM'  and  NN’  are  the  two  world  segments  corresponding  to  the  lowest  image  segments  of  the 
image.  Their  midpoints  are  P  and  Q.  One  of  the  ends  of  the  initial  cross-segment  is  taken  to  be  one 
of  these  two  midpoints,  here  Q,  and  its  direction  is  the  normal  to  the  corresponding  edge  segment, 
NN',  in  the  ground  plane.  If  MM'  and  NN'  are  found  to  be  parallel  (within  acceptable  limits),  the 
other  end  of  the  initial  cross-segment  is  taken  to  be  the  intersection  R  of  this  normal  with  the  other 
edge  segment,  MM'.  If  the  two  edge  segments  are  not  parallel,  as  can  happen  in  a  turn  (Figure 
1 1),  a  kind  of  center  of  curvature  C  of  the  turn  is  found  as  the  intersection  of  the  normals  to  the 
midpoints  of  the  two  segments,  and  the  width  w  of  the  road  is  calculated  as  the  difference  between 
CQ  and  CP.  Then  the  endpoint  R  of  the  initial  cross-segment  is  taken  at  a  distance  w  from  its 
endpoint  Q. 
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7.  Experimental  evaluation  of  the  proposed  zero-bank  algorithm 


For  experimental  evaluation  of  the  proposed  method,  a  world  road  is  given  analytically.  The 
centerline  describes  an  "S"  on  a  hill  or  in  a  valley,  and  the  road  surface  is  defined  by  horizontal 
cross-segments  cutting  the  centerline  at  their  midpoints.  The  analytic  expressions  for  the  edges  are 
derived  from  the  following  specifications; 

The  road  centerline  projects  on  the  horizontal  plane  as  a  straight  part  of  length  0.5  units, 
followed  by  a  90  degree  right  turn  with  radius  1,  then  a  90  degree  left  turn  with  radius  1.  The  road 
centerline  is  on  a  hill  or  in  a  valley  described  by  one  period,  trough  to  trough  or  peak  to  peak,  of  a 
sinusoidal  cylinder.  The  road  edges  are  deduced  from  the  centerline  by  taking  distances  of  0.2  units 
on  each  sides  of  horizontal  normals  to  the  centerline  curve.  The  slope  starts  at  the  beginning  of  the 
road(straight  part)  and  changes  sign  when  the  road  is  at  90  degrees  from  its  starting  direction.  On 
Figtires  12  to  16,  a  top  view  (center)  and  a  side  view  (right)  of  this  road  in  orthogonal  projection 
are  drawn  on  the  right. 

Five  slope  configurations  were  tested: 

a.  Horizontal  road. 

b.  Hill  height  0.1. 

c.  Hill  height  0.2. 

d.  Valley  depression  0.1. 

e.  Valley  depression  0.2. 

The  slope  changes  can  be  seen  on  the  side  view  of  the  road,  at  the  extreme  right  of  Figures  12  to 
16. 

The  perspective  image  of  this  road  is  computed,  assuming  that  the  image  plane  is  vertical,  and 
that  the  focal  point  is  at  a  height  1,  at  a  horizontal  distance  0.2  from  the  start  of  the  road.  It  is 
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shown  on  the  left  of  Figures  12  to  16. 


The  algorithm  was  run  on  this  perspective  image,  and  the  world  road  that  the  algorithm 
reconstructed  is  shown  in  top  view  and  side  view  for  comparison  with  the  actual  road.  For 
debugging  purposes,  the  images  of  the  inverse  perspectives  of  the  cross-segments  are  also  shown 
on  the  road  image  (their  end  points  must  be  on  the  road  edge  images  ). 

Three  algorithms  were  tested: 

-  The  flat  earth  assumption  algorithm 

-  A  vanishing  point  method 

-  The  proposed  zero-bank  road  algorithm 

The  flat  earth  assumption  algorithm  is  the  simplest.  The  camera  is  assumed  to  look  at  a  flat 
world.  In  the  case  of  a  camera  on  a  vehicle,  the  plane  of  the  world  is  an  extension  of  the  planar 
patch  supporting  the  vehicle  at  the  time  of  image  acquisition,  and  is  calculated  from  the  camera 
height  and  tilt.  The  central  projection  of  the  edges  of  the  road  image  on  this  plane  is  assumed  to 
give  the  edges  of  the  world  road. 

A  vanishing  point  method  was  briefly  described  in  [1].  In  the  introduction,  we  described  how 
such  a  method  can  construct  two  parallel  segments  in  space  from  two  image  segments  (Figure  1), 
and  we  mentioned  that  one  of  the  problems  was  the  choice  of  segments  in  the  persective  image 
which  are  images  of  opposite  segments  (segments  containing  opposite  points  of  the  road).  The 
implementation  tested  here  was  based  on  [1]  and  on  discussions  with  Eli  Liang.  It  uses  the  world 
segments  calculated  at  the  previous  iteration  step  to  deduce  the  positions  of  the  images  of  opposite 
segments  at  the  new  step.  At  an  intermediate  iteration  step,  once  two  parallel  world  segments  are 
found,  a  plane  normal  to  these  segments  is  calculated  so  that  it  goes  through  the  far  end  of  one 
segment  and  cuts  the  other  segment.  The  end  of  one  segment  and  the  intersection  of  the  normal 
plane  with  the  other  are  assumed  to  form  a  pair  of  opposite  points,  and  the  segments  in  the  image 
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which  follow  the  images  of  these  two  points  are  assumed  to  be  images  of  parallel  segments  and  are 
used  in  the  next  iteration  step.  This  approximation  therefore  takes  opposite  points  where  they  can 
be  calculated  (from  the  road  element  just  calculated)  and  assumes  that  the  segments  coming  just 
after  the  images  of  these  opposite  points  on  the  perspective  image  are  images  of  parallel  segments. 
As  discussed  in  the  next  section,  this  approximation  is  not  very  satisfactory. 

8.  Qualitative  evaluation  of  the  results 

The  flat  earth  assumption  algorithm  performs  better  than  the  proposed  method  on  flat  ground 
(Figure  12).  For  hill  or  valley  configurations  (Figures  13  to  16),  the  proposed  zero-bank  road 
algorithm  gives  a  much  more  correct  inverse  perspective  of  the  road  than  the  other  two  algorithms. 
The  drawings  also  show  how  far  from  the  truth  the  flat  earth  assumption  can  be  as  soon  as  there  are 
some  slope  changes  in  front  of  the  camera. 

The  vanishing  point  method  attempts  to  construct  a  developable  ribbon.  As  a  result  it 
underestimates  the  first  turn.  Further  down  the  road,  the  catastrophic  deterioration  of  its 
reconstmction  is  due  to  the  added  errors  from  the  approximation  for  finding  the  images  of  parallel 
segments . 

9.  Concluding  remarks 

Modeling  roads  as  Brooks  ribbons  in  space  with  horizontal  generators  of  constant  length,  we 
have  shown  that  two  neighboring  generators  form  approximately  a  warped  isosceles  trapezoid.  It 
follows  that  when  one  generator  is  known,  the  next  generator  can  be  recovered  completely  if  one  of 
its  ends  is  given.  On  the  basis  of  this  observation,  we  proposed  an  iterative  algorithm  for 
reconstructing  the  a  world  road  from  its  perspective  image.  At  an  intermediate  step  of  the  iteration, 
a  previous  generator  is  known,  and  a  point  on  one  road  edge  image  is  chosen  as  the  image  of  one 


end  of  the  next  generator.  This  end  has  only  one  degree  of  freedom  along  the  line  inverse 
perspective  of  its  image.  From  our  previous  observation,  the  other  end  also  has  only  one  degree  of 
freedom.  Therefore  its  locus  is  an  arc  and  depends  on  one  parameter.  We  calculate  the  parameters 
such  that  this  other  end  has  its  image  in  the  image  of  the  other  edge  of  the  road,  as  solutions  of 
cubic  equations.  When  more  than  one  solution  occurs,  we  pick  the  appropriate  generator  on  the 
basis  of  minimal  change  of  slope  of  the  resulting  road. 

We  tested  this  algorithm  on  a  road  image  synthesized  from  an  analytic  description  of  a  road 
describing  a  "S" ,  on  a  hiU  and  in  a  valley.  With  this  algorithm  the  results  of  the  reconstruction  are 
much  closer  to  the  original  road  than  with  two  other  algorithms  chosen  for  comparison,  an 
algorithm  based  on  a  flat  earth  assumption,  and  an  algorithm  based  on  vanishing  points.  In  the 
introduction,  we  summarized  the  limitations  of  vanishing  point  methods  for  road  inverse 
perspective. 

Our  next  goal  is  to  test  this  algorithm  on  images  of  real  roads  and  to  check  whether  in  natural 
conditions  warped  isosceles  trapezoids  perform  better  than  planar  patches. 
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